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LOW  ALTITUDE  SIMULATOR  TRAINING:  A-IO  AIRCRAFT 


I.  INTRODUCTION 

This  report  is  organized  in  the  following  manner.  This  section  (Introduction)  reviews  background  material,  describes 
the  experimental  task  selection  process,  provides  a  description  of  these  tasks,  and  presents  the  objectives  of  the  effort. 
Section  II  describes  the  experimental  training  apparatus  and  presents  the  general  approach  used  for  study.  To  facilitate 
operational  scheduling  and  experimental  design  requirements,  the  effort  was  separated  into  two  phases.  Laeh  phase  is 
reported  as  a  separate  unit.  Phase  I  described  in  Section  111,  focuses  on  experimental  issues  concerning  training  of 
eleinentarv  basic  attack  maneuvers  (HAMs).  Phase  II  deals  exclusively  with  low  level  navigation  (I.LN)  training  issues 
and  is  described  in  Section  IV.  Subjects,  experimental  training  procedures,  dependent  measures,  data  analysis 
procedures,  and  results  are  explainer!  for  each  phase  separately  in  their  respective  sections.  Section  V  presents  discussion 
of  the  results  obtained  from  both  phases.  Finally,  conclusions  and  recommendations  are  presented  in  Section  VI. 


Background 

Tactical  requirements  to  fly  low  altitude  operations  in  any  terrain  and  in  any  geographical  area  have  been  difficult 
to  satisfy.  The  A- 10  aircraft  evaluation  reports  from  Bentwaters  AFB  and  Myrtle  Beach  AFB  have  shown  that  transition 
training  B-eoiirse  graduates  typically  have  a  minimum  6-month  orientation  period  before  they  are  considered  ready  to 
fly  lead  on  low  altitude  operations.  Data  from  actual  combat  situations  suggest  that  orientation  to  novel  low  altitude 
environments  may  be  a  persistent  problem  for  the  first  10  sorties  (Joss,  1978).  The  use  of  simulators  for  training  low 
attitude  operations  represents  an  attractive  possibility  as  a  means  for  reducing  these  lengthy  orientation  periods. 

Simulator  research  and  development  involving  low  altitude  operations  has  focused  primarily  on  visual  cue 
requirements.  Current  computer-generated  imagery  (CGI)  capabilities  preclude  simulation  of  highly  detailed  low  altitude 
env  ironinenls.  V  arious  coplanar  and  vertical  pattern  configurations  have  been  evaluated  in  an  attempt  to  optimize  existing 
CGI  capabilities.  Reviews  of  this  work  can  be  found  in  Edwards,  Pohlman,  Buckland.  and  Stephens  (1981)  and  Martin 
and  Kinaldueci  (in  press).  The  use  of  existing  CGI  capabilities  in  the  development  of  operational  scenarios  for  training 
low  altitude  procedures  is  an  area  of  research  not  previously  addressed.  Whether  simulator  to  aircraft  transfer  benefits 
can  be  derived  from  such  scenarios  is  an  issue  of  major  interest  to  operational  program  managers.  The  current  effort 
addressed  this  issue  by  examining  the  effectiveness  of  simulator  scenarios  developed  to  train  elementary  BAM  and  I.LN 
tasks.  The  effort  was  conducted  in  conjunction  with  an  ongoing  A- 10  Advanced  Simulator  for  Pilot  Training  (ASPTl 
conversion  and  surface  attack  training  program  designed  for  initial  entry  students  transitioning  to  the  A- 10  aircraft.  The 
experiment  was  designed  to  have  minimal  impact  on  regular  operations  of  the  A- 10  training  program. 


Experimental  Tasks  Selection 

The  selection  of  experimental  tasks  was  based  on  two  requirements,  lirst.  the  tasks  had  to  be  normally  trained 
in  the  aircraft  at  low  to  medium  altitudes.  The  rationale  supporting  this  requirement  was  the  research  need  to  address 
transfer  effectiveness  issues  of  tactical  low  altitude  training  in  the  simulator.  Second,  to  properly  evaluate  the  training 
effectiveness  of  the  A  SIT  while  minimizing  scheduling  conflicts  with  regular  squadron  operations,  it  was  necessary  to 
select  tasks  which  were  initially  practiced  in  the  aircraft  shortly  after  scheduled  ASPT  sessions.  Elementary  BAM  and 
I.LN  tasks  fulfilled  these  requirements.  Both  tasks  offered  operations  in  the  low  to  medium  altitude  regime  and  both 
tasks  were  initially  trained  in  the  A- 10  aircraft  shortly  after  the  scheduled  ASPT  training  sessions. 


Experimental  Tasks  Description 

Klrmrntury  HAM.  Elementary  BAMs  are  the  building  blocks  used  in  practically  all  surface  attack  maneuvers. 
Eleinentarv  BAMs  consist  of 90°/ 1 80°  low  altitude  turns,  .10° and  20° dive,  roll-in  and  recovery  (l)RR)  tasks,  and  target 
tracking  exercises.  The  purpose  of  elementary  BAMs  is  to  familiarize  the  student  pilot  with  the  use  of  the  head-up  display 


(HUD)  ami  provide  experienee  in  surface  attack  procedures.  Specifically,  'X >°/ 1 80°  turns  are  designed  to  train  the  pilot 
to  execute  a  turn  at  a  prescribed  angle  of  attack  (ADA),  to  maintain  a  presc  ribed  altitude,  to  roll  out  approximately  the 
prescribed  number  of  degrees  from  the  initial  heading,  and  to  maintain  situation  awareness.  The  20°  and  TO0  DRR  tasks 
are  designed  to  acquaint  the  pilot  with  techniques  required  for  surface  attack  maneuvers.  Additionally,  the  pilot  is  taught 
to  establish  a  cross  check  of  HUD  data  to  include  dive  angle,  airspeed,  and  altitude.  Target  tracking  is  required  for 
gun  employment.  The  objective  of  this  exercise  is  to  track  the  gun  cross  up  to  a  target,  hold  it  there  momentarily,  and 
be  aware  of  parameters  displayed  in  the  HU  D. 

IX. V.  The  objective  of  the  LLN  training  mission  is  to  develop  the  basic  skills  necessary  to  perform  tactical  navigation. 
Each  LLN  mission  involves  the  review  and  execution  of  a  predetermined  navigation  exercise  to  a  target  or  practice  area. 
The  LLN  route  consists  of  a  number  of  sections  or  legs.  The  student  is  provided  topographic  maps  of  each  leg  with 
headings,  times,  and  distances  clearly  indicated.  The  LLN  training  mission  is  designed  to  instruct  the  student  in  the 
fundamentals  of  low  level  operations  and  dead  reckoning  navigation.  Low-level  fundamentals  include  route  study,  route 
timing,  route  entry,  flight  formation,  radio  procedures,  navigation  error  analysis,  and  aircraft  control  techniques.  Dead 
reckoning  consists  of  flying  from  one  point  to  another  relying  solely  on  time,  speed,  and  direction.  To  successfully  master 
dead-reckoning  procedures,  the  pilot  must  be  proficient  at  map  reading,  identifying  visual  checkpoint  features,  and 
determining  the  effect  wind  has  on  the  aircraft  ground  track.  LLN  training  given  to  the  A- 10  B-course  pilot  focuses  on 
those  aspects  of  the  mission  peculiar  to  the  A- 10  weapon  system.  After  completion  of  this  training,  the  pilot  is  taught 
more  advanced  operations  involving  low  altitude  tactical  navigation  and  low  altitude  tactical  formation. 


Objectives 

There  were  two  primary  objectives  of  this  effort:  (a)  to  evaluate  the  effectiveness  of  AS  FT  scenarios  to  train 
elementary  BAMs,  and  (b)  to  evaluate  the  effectiveness  of  ASPT  scenarios  to  train  I.L.N.  A  transfer-of-training  paradigm 
was  used  to  address  these  objectives.  Training  effectiveness  was  operationally  evaluated  In  examining  trends  in  simulator 
performance  data  and  by  examining  differences  between  experimental  and  control  group  performances  on  tasks  performed 
in  the  aircraft. 

A  secondary  objective  to  the  experiment  was  to  evaluate  the  sensitivity  of  the  aircraft  analysis  variables  in  the 
detection  of  performance  differences  between  experimental  and  control  groups.  IP  evaluations  were  the  only  metrics 
available  for  assessing  BAM  and  LLN  student  aircraft  performances.  Objective  measures,  such  as  bomb  scores,  were 
not  obtainable  by  the  very  nature  of  these  tasks.  All  aircraft  performance  analysis  variables  used  in  this  effort  were, 
therefore,  based  on  IP  evaluation  data.  The  sensitivity  of  IP  evaluations  to  between  group  performance  variations  has 
been  shown  to  be  much  less  than  the  sensitivity  of  more  objective  measures  of  performance  for  weapons  delivery  tasks 
(Gray  &  Fuller.  1977).  Since  this  was  seen  as  a  potential  problem,  |>ost  hoi  evaluations  of  statistical  power  were  used 
to  operationally  evaluate  the  sensitivity  of  aircraft  performance  variables  used  in  the  analyses. 


II.  GENERAL  APPROACH 

The  apparatus  and  methodology  descriptions  presented  in  this  section  arc  generic  to  lioth  BAM  and  LLN  phases. 
The  methodology  concerns  that  are  specific  to  the  individual  phases  are  presented  in  their  respective  sections. 


Training  Apparatus 

The  A- 10  ASPT  was  used  in  this  experiment  to  train  experimental  group  subjects  BAM  and  I.l.N  exercises.  A 
complete  description  of  the  A- 10  ASPT  system  can  be  found  in  Gray.  Chun.  Warner,  and  Kubanks  (1981 1  and  a  summary 
of  recent  modifications  is  included  in  Brooks  and  Lyon  (in  press).  The  g-sCat  and  g-suit  force  cueing  systems  on  the 
ASPT  were  used  in  this  effort;  however,  the  ASPT  platform  motion  system  was  nul.  The  g-suit  used  in  the  experiment 
was  the  standard  g-suil  used  in  the  A- 10  aircraft,  lnflation/deflation  of  the  suit  was  computer  controlled  and  simulated 
the  suit  pressures  present  during  actual  aircraft  maneuvers.  The  visual  system  on  the  ASPT  consisted  of  seven  36-inch 
monochromatic  cathode  ray  lubes  (CRTs)  placed  around  the  cockpit  to  allow  the  pilot  +  1 10°  to  —  40°  vertical  cueing 
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ami  *  I  .">0°  horizontal  cueing.  BAM  and  LLN  visual  environments  were  developed  applying  experience.  research 
findings,  anil  inferences  from  irevious  ASPT  CGI  development  efforts. 

Method 

\ s  pail  ol  the  regular  training  syllahus,  A- 10  B-course  student  pilots  received  two  training  sessions  in  the  ASPT. 
The  first  session,  consisting  ol  two  1.5-hour  sorties,  was  used  to  train  conversion  tasks;  surface  attack  maneuvers  were 
trained  in  the  second  session,  which  consisted  of  three  1.5-hour  sorties.  Appendix  A  contains  the  A- 10  ASPT  syllabi 
used  at  the  time  of  this  experiment.  The  two  training  sessions  were  scheduled  approximately  3  weeks  apart.  During  this 
3-week  period,  students  received  conversion  phase  training  in  the  aircraft,  along  with  scheduled  academics.  Students 
typically  Hew  their  first  BAM  aircraft  sortie  within  1  week  following  ASPT  conversion  training.  It  took  approximately 
lOdavs  for  all  students  within  a  class  to  complete  the  three  required  BAM  airrrafl  sorties.  Two  aircraft  sorties  containing 
l.l.N  exercises  were  scheduled  within  1  week  following  ASPT  surface  attack  training.  The  delay  between  training  in  the 
ASPT  and  the  first  of  the  two  LLN  aircraft  sorties  was  typically  3  to  4  days.  The  separation  of  BAM  and  I. LIS  operations 
into  two  phases  allowed  independent  group  assignment  and  data  analysis  procedures.  This  design  had  minimal  impact 
on  training  operations  and  was  acceptable  to  operational  training  personnel.  Briefings  for  both  BAM  and  LLN  ASPT 
sorties  were  delivered  by  A-10  IPs.  The  same  three  A-10  IPs  alternated  briefing  and  instructing  students  on  research 
sorties  throughout  the  effort.  On  each  sortie,  students  received  conventional  instruction  and  feedback  from  the  IP.  Except 
for  the  students  themselves  and  the  research  IPs,  student  group  assignments  were  unknown  to  squadron  personnel. 

III.  PHASE  I -HAM 

Subjects 

The  subjects  were  42  A-10  B-course  student  pilots  assigned  to  the  355th  Tactical  Training  Wing,  Davis- Monthau 
A  KB.  served  as  subjects.  All  subjects  were  recent  l  ndergraduate  Pilot  Training  ll  PTl  graduates  and  had  completed 
lighter  lead-in  training,  but  had  not  been  qualified  in  any  other  operational  aircraft.  As  part  of  the  normal  syllabus,  each 
subject  was  a  participant  in  the  A-10  ASPT  training  program.  For  study  purposes,  these  student  pilots  were  randomly 
assigned  to  either  an  ex[ierimenlal  or  a  control  group:  22  were  assigned  to  the  experimental  group,  and  the  remaining 
20  served  as  controls. 

Experimental  Training 

l  poo  completion  ol  ASPT  conversion  training,  experimental  group  subjects  received  a  briefing  that  covered  the 
visual  environment,  task  requirements,  and  procedures  for  the  BAM  experimental  sortie.  After  this  briefing,  subjects 
flew  the  1  -fiour  HAM  sortie  on  the  ASPT  with  an  A-10  IP  providing  instruction  from  the  main  console.  The  sortie  was 
developed  with  the  help  of  IPs  from  the  A- 10  operational  training  development  section  at  Davis-Monthan  AF  B.  The  sortie 
begun  with  a  takeoff  followed  by  a  5-minute  area  orientation  period.  Subjects  then  performed  each  task  a  set  number 
of  trials.  A  detailed  syllabus  of  the  BAM  sortie  is  provided  in  Table  1.  Due  to  the  nature  of  the  tasks  being  trained, 
the  experimental  sortie  was  flown  in  a  continuous  fashion;  at  no  time  during  the  training  period  were  the  simulator  freeze 
or  reset  functions  used  All  control  group  subjects  received  a  1-hour  ASPT  "control  "  sortie  following  ASPT  conversion 
training  to  equate  the  groups  with  respect  to  total  time  in  the  simulator.  The  control  sortie  was  simply  a  1-hour  repeat 
of  the  last  sortie  flown  during  ASIT  conversion  training)  Appendix  A  I. 


Table  l .  Detailed  ASPT  BAM  Syllabus 


lu  Takeoff; Climb;  Level  off  (u  bOOOIeel  ADI. lwx:2(XX)/3) 

I  li  Area  Orientation  (5  minutes) 

2a  90°/ 180° Turns  ( 17.3  units  AOA)  -  (312311  repetitions) 

2b  90<7180°Tums(21.5umts  AOA)- (3123R repetitions) 

3  30°  Dive,  Roll-In  and  Reeovery  (7  repetitions) 

4  20°  Dive,  Roll-In  and  Reeovery  (7  repetitions) 

5  Target  Tracking (3  repetitions) 


The  computer-generated  visual  environment  used  for  ASPT  BAM  training  during  phase  I  was  an  area  approximately 
Q  miles  by  9  miles  (Figure  1).  In  this  area  were  plaeed  several  mountain  ranges,  ranging  in  height  from  3500  feet  to 
5500  feet  above  ground  level  (  AGI.I.  with  scattered  2000-foot- A GI,  pinnacles.  These  mountains  surrounded  a  gaming 
area  approximately  3  miles  by  3  miles  in  size.  Within  this  gaming  area  were  50-foot  inverted  ’  etrahedrons  used  to  provide 
vertical  development  cues  at  low  altitudes.  These  cues  covered  the  entire  gaming  area  with  1000-foot  separation.  All 
BAM  tasks  were  practiced  within  this  gaming  area.  For  the  target  tracking  task,  one  of  several  tank-shaped  targets  in 
the  area  was  made  twice  its  normal  size  and  placed  upon  a  100-foot-square  white  background.  This  was  done  to  guarantee 
ease  in  locating  the  correct  target.  None  of  the  other  targets  located  in  the  area  had  any  special  puqxise  in  this  sortie. 
A  runwav  that  was  inserted  into  the  environment  at  the  beginning  of  the  sortie  was  used  for  takeoffs.  After  the  simulator 
indicated  airborne  at  2000  feet  AG1..  the  visibility  of  the  gaming  area  was  reduced  to  3  miles,  and  the  runway  was 
removed.  AlOOOOfeet  AGl..  unlimited  visibility  was  restored  over  the  gaming  area,  and  the  exercise  began. 


Figure  /  Map  representation  of  the  ASPT  BAM  visual  environment. 


Dependent  Meanurf* 


\SPT  Performance  Measurement.  H  AM  .simulator  performance  ilala  were  collected  on  all  experimental  subjects 
using  tin1  simulator's  automated  |>crformaucc  measurement  system  (PMS|.  A  description  ol  the  ASPl  automated  PMS 
can  In-  found  in  < Iras  et  al.  (I‘WI|.  The  measures  collected  for  each  BAM  task,  along  with  the  sturt/stop  parameters 
used  to  define  data  collection  periods,  are  listed  in  I  able  2. 

Table 2.  ASPT  BAM  Performance  Measures 


Scoring  Logic 

Training  Kirrcitf 

Start 

Stop 

Performance  Mranurc# 

000M80o  Turn* 

ABS  Bank 

ABS  Bunk 

KMS  deviation*  from  desired  AOA 

1  AOA  =  I7.5&21.5) 
(Altitude  =  .AIK)  leet  1 

<;t  20° 

I  T  5° 

KMS  deviations  from  desired  Altitude 

30°  and  20°  DKK 

50°  event: 

KMS  deviation*  from  desired 

ABS  Bank 

Vertical 

Dive  Angle  on  Final 

IT  10°; 

Veloeitv 

KMS  G-loadingon  Final 

Dive  Angle 

GT0 

Pickle  Parameters  (captured  (a 

<;t  20° 

2500  feet  AGl.  for  50° and 

20°  event: 

1800  feet  AGI.  for 20"): 

ABS  Bank 

\  ertical 

Bank  Angle 

IT  10°; 

\  eloeitv 

Dive  Angle 

Dive  Angle 

CTO 

Knots  Indicated  Airs[>oed 

GT  1 5° 

G  loading 

Target  Tracking 

Within  1800  feet 

Altitude 

Hit/Miss  on  target  (captured 

of  Target: 

I  T  MOO  feel 

between  1600  Icct  &  1400  feet  AGL) 

Heading  Between 

Alii. 

RMS  miss  distance  from  target  feet 

+  1  1 0°  &  - 1 50°; 
ABS  Bank 

(between  1600  feet  &  1400  feet  AGF) 

I  T  20" 

Ain  raft  Performance  Measurement.  Student  performance  in  the  aurrult  was  evaluated  using  data  obtained  from 
student  mission  gradesheets  and  HAM  performance  questionnaires  developed  lor  this  cllurt  (  Appendix  B.  Table  It- 1 1. 
Plight  IPs  completed  student  gradesheets  and  performance  questionnaires  at  the  debriefing  of  each  flight.  Both 
measurement  instruments  used  a  5-point  rating  scale.  This  rating  scale  was  the  standard  0  to  4  scale  used  within  T  AC. 
for  evaluating  student  performances:  0  ratings  indicated  lack  of  ability  or  knowledge;  1  ratings  indicate  safe  but  limited 
proficiencv;  2  ratings  indicated  performance  was  essentially  correct  with  errors  identified  and  corrected;  3  ratings 
indicated  performance  was  correct,  efficient,  skillful  and  without  hesitation;  and  ratings  of  1  reflected  performance  of 
an  unnsuallv  high  degree  of  ability.  Hating  criteria  for  individual  tasks  were  specified  in  the  A-IO  flight  Objectives 
Pamphlet.  All  B-eourse  conversion  phase  sorties  were  flown  as  two-slup  flights  composed  ol  one  student  with  one  IP. 

I  toe  to  the  operational  environment  ami  non-interference  status  of  the  effort.  llVstudenl  assignments  could  not  lie 
experimenlallv  controlled.  Assignments  were  usually  based  on  IP  availability  and  student  progress. 

'I’lie  student  mission  gradesheets  were  used  to  assign  an  overall  rating  to  each  task  performed  on  a  flight.  The  A- 10 
B-eourse  svlluhus  contained  three  sorties  on  which  H  AM  tasks  were  practiced.  ( .radeslieel  ratings  assigned  to  H  AM  tasks 
performed  on  these  sorties  were  recorded  lor  later  analvsis.  Individual  re|>etitions  data  were  obtained  using  the  B  AM 
performance  questionnaire.  I  his  questionnaire  wus  designed  to  obtain  measures  similar  to  those  obtained  from  the  ASPT 
automated  PMS.  I  sing  I  hr  questionnaire,  II’*  provided  overall  ratings  on  each  BAM  task  rr|>etition  |X‘rformed.  In 
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addition,  maximum  altitude  deviations  from  desired  altitude  wore  recalled  for  each  17.5  and  21.5  unit  AOA  turn 
performed.  IPs  also  assigned  AOA  control  ratings  to  each  turn  repetition.  For  20°  and  30°  DRR  tasks,  IPs  recalled 
deviations  in  airs|x-cd  und  tlive  angle  from  desired  parameters  at  pickle  altitude  for  each  repetition.  Not  ail  BAM  tasks 
were  performed  on  each  of  the  three  HAM  aircraft  sallies.  Tallied  lists  the  tasks  that  were  practiced  during  each  flight. 


Tabled.  Distribution  of  BAM  Tasks 


Sortie 

TASK 

Turn* 

20°  DRR 

30°  DRR 

Target  Tracking 

CV-t 

Yes 

No 

Yes 

Yes 

t:\-5 

Yes 

Yes 

No 

No 

t:\-6 

Yes 

Yes 

Yes* 

No 

Squadron  instructor  pilots  (IPs)  we  briefed  on  the  purpose  and  requirements  of  Phase  I  of  the  program. 
Additionally,  a  letter  outlining  the  data  collection  requirements  was  circulated  among  the  participating  squadrons  prior 
to  each  data  collection  period.  During  these  periods,  a  researcher  was  on  site  to  administer  questionnaires,  collect 
completed  data  collection  materials,  and  answer  questions  pertaining  to  the  effort.  Despite  these  efforts,  aircraft 
performance  data  for  two  experimental  subjects  were  removed  from  final  analysis  due  to  IP  misunderstandings  of  data 
requirements.  No  problems  were  encountered  with  the  aircraft  performance  data  collected  for  the  20  control  subjects 
and  all  of  these  subjects  were  used  in  the  final  analyses. 


Data  Analy  sis  Procedures 

Simulator  performance  data  analyses  focused  on  identifying  and  describing  significant  changes  in  performance  as 
a  function  of  training.  Single  factor  with  repeated  measures  (A  x  S)  Analysis  of  Variance  (ANOVAl  procedures  (Keppel. 
1973)  were  used  to  analyze  all  intratrial  RMS  deviation  data  obtained  for  each  task.  The  same  ANOV'A  procedures  were 
used  to  analyze  transformed  values  of  release  altitude  parameters  for  20°  and  30°  (DRR)  exercises.  Post  hoe  comparisons 
on  trial  means  were  made  using  linear  trend  analyses  procedures  (Keppel,  1973)  for  variables  whose  ANOYAs  were 
found  to  be  significant  at  the  p  <  .05  level. 

Aircraft  performance  data  analyses  examined  between  group  differences  and  identified  those  variables  most 
sensitive  to  group  performance  variations.  Aircraft  performance  data  obtained  from  BAM  performance  questionnaire  and 
student  mission  gradesheets  were  sorted  into  individual  analysis  variables  for  statistical  examination.  Selection  of 
analysis  variables  for  the  various  tasks  was  based  on  a  requirement  to  achieve  compatibility  with  the  simulator 
performance  analysis  variables.  In  addition,  it  was  desirable  that  these  aircraft  performance  variables  be  similar  to  those 
outlined  in  test  plans  for  the  test  and  evaluation  of  future  Tactical  Air  Command  (TAC)  weapon  system  trainers.  Single 
factor  A  NOVA  procedures  (Keppel,  1973)  were  used  to  test  between  group  differences  of  each  analysis  variable.  Because 
the  use  of  parametric  procedures  to  analyze  subjective  evaluations  might  be  debated,  analysis  variables  were  also  tested 
using  the  nou-parametric  Mann-Whitney  L-statistio  (Siegel,  1956).  Finally,  post  hoc  determinations  of  statistical  power 
( Keppel.  1973)  were  computed  for  each  variable  to  assess  the  sensitivity  of  the  measures  to  group  performance  variations. 


Phase  I  Results 

Results  of  the  analyses  of  simulator  and  aircraft  performance  data  arc  presented  separately  ,  with  simulator  data 
analyses  reported  first. 
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ASPT  T rend  Analyses 


I  7.5  iiml  1V)..>  l  nil  -W  M  Turns.  Two  measures  of  training  performance  Vv rrr  derived  (rum  the  data  for  hoth  17.5 
and  21.5  unit  I  AO  A I  turns;  thus,  a  total  ol  lour  measures  were  extracted.  For  each  event,  the  two  measures  were  intratnal 
RMS  ADA  deviation  ami  mlralrial  R  MS  altitude  dev  iation. 

Post  hoe  examination  of  the  data  revealed  that  the  start/stop  logic  used  to  determine  data  collection  “window"  periods 
had  luased  the  AO  A  deviation  calculations  toward  higher  RMS  values.  At  roll-in  and  roll-out,  when  AOA  is  typically 
tar  hclow  the  desired  value,  AOA  deviations  were  erroneously  collec  te  d  for  an  undetermined  number  of  iterations  and 
incorporated  in  intratrial  RV1S  calculations  (review  scoring  logic  in  Table  2).  The  effect  this  had  on  resulting  AOA  RMS 
deviation  values  was  not  consistent;  therefore,  these  data  were  not  analyzed.  To  the  c  xlent  that  the  most  critical  periods 
to  control  altitude  during  a  turn  are  at  roll-in  and  roll-out,  the  w  indow  logic  was  deemed  appropriate  for  examining  altitude 
control.  The  analyses  of  the  training  performances  for  17.5  and  21.5  unit  AOA  turns  was  therefore  restricted  to  intratrial 
R  MS  altitude  dev  iation  data. 

-ANOV  As  we  re  computed  on  intratrial  RMS  altitude  deviation  data  for  the  17.5  and  21.5  unit  AOA  turn  events. 1 
Results  for  the-  I  7. 5  unit  AOA  event  were  significant  ( Fw  =  2.22;  p  <.05i.  Linear  trend  comparisons  on  trial  means 
also  produced  significant  results  (1- 1.20.0  =  9. 1 ;  p  <  .001 1;  all  other  orthogonal  trend  1  -omponents  were  not  significant. 
These  results,  combined  with  an  examination  of  trial  means  (Appendix  B,  Table  B- 1 ),  show  a  linear  decrease  in  altitude 
deviation  as  a  function  of  trials. 

The  A  NOV  A  computed  for  21.5  AOA  turns  data  was  not  significant.  Examination  of  Appendix  C,  Table  C-l  shows 
less  than  a  bO-foot  range  among  trial  mean  deviations.  Additionally,  trial  means  fall  far  below  those  initially  obtained 
for  the  17.5  AOA  event  even  though  the  21.5  AOA  event  is  considered  by  pilots  to  be  more  demanding.  The  21.5  AOA 
event  was  trained  immediately  following  the  17.5  AOA  event:  therefore  carry-over  effects  may  lie  partly  responsible  for 
the  inability  to  demonstrate  improved  performances  as  a  function  of  trials. 

■i0°  DRR.  Two  sets  of  variables  were  extracted  from  data  for  the  50°  DRR  exercise.  The  first  set  consisted  of  RMS 
dive  angle  deviation  and  RMS  g  values  computed  for  “on-final”  parameters.  On-final  for  a  DRR  task  was  defined  as 
the  period  from  roll-out/wings-level  to  when  the  aircraft  achieves  weapons  release  altitude.  The  second  set  of  variables 
consisted  of  release  altitude  (pickle)  values  of  dive  angle,  airspeed,  g,  and  bank  angle  transformed  to  estimated  bombing 
impact  error  distances.  These  transformations  ignored  pipper  placement  error  because  specific  pipper  placement  was 
not  a  task  requirement.  In  addition,  the  transformations  were  based  on  the  trajectory  of  a  low  drag  bomb. 

.ANOVAs  were  -(imputed  for  both  of  the  on-final  parameter  values  from  the  first  set  of  variables.  Results  for  dive 
angle  deviation  values  showeil  significant  differences  across  trials  |F(),9~  =  2.2b:  p  <  .051.  The  linear  component  of 
the  trend  comparisons  computed  for  trials  was  significant  (F|  ,  =  9. 1 1 ;  p  <  .0011;  all  other  orthogonal  trend 

components  were  not  significant.  These  results  combined  with  an  examination  of  trial  means  (Appendix  C.  Table  C-l  I 
show  significant  decreases  in  RMS  dive  angle  error  as  a  function  of  trials.  The  results  of  the  A  NOV  A  computed  for  on-final 
RMS  g  values  were  not  significant.  The  RMS  g  value  trial  means  (Appendix  C,  Table  C-l  I  showed  fairly  good  control 
of  g  over  all  trials.  Thus,  little  improvement  could  be  expected. 

Rickie  parameter  values  were  transformed  to  equivalent  miss  distances,  in  leet,  to  enable  arithmetical  computations 
and  comparisons  among  the  four  variables.  In  dive  bomb  exercises,  errors  can  cancel  each  other,  thus  minimizing  overall 
bombing  error,  for  toe  DRR  tasks,  however,  the  objective  was  not  to  compensate  but  to  minimize  all  parameter  errors. 
Absolute  values  of  the  transformations  were  therefore  used  in  deriving  analysis  variables.  ANOV  As  were  computed  for 
transformed  values  oi  each  parameter  with  additional  ANOVAs  computed  on  two  combinations  of  the  transformed  values. 
The  first  combination  was  a  summation  ol  dive  angle,  airspeed,  and  g  error  transformations  to  obtain  a  total  longitudinal 
deviation  value,  I  hr  second  combination  was  a  summation  of  dive  angle,  airs|>eed.  g.  and  bank  angle  error 
transformations  to  obtain  a  total  system  deviation  value.  ANOV  A  results  lor  the  transformed  data  were  significant  for 
dive  angle  deviations  (F(1  =  5.08:  p  <  .025).  airspeed  deviations  (F,.  |ov  =  2.95;  p  <  .025).  total  longitudinal 


1  Doe  In  missing  <  hsrrvalions.  trials  II  and  1 2  were  deleted  from  the  analvsrs  of  I  7.5  and  2 1 .5  AOA  data.  Remaining  musing 
ril>»M»rvat:<*n««  lor  this  and  following  analyse*  were  calculated  using  the  procedure  desrrihed  m  Kirk  ( p.  1 46|. 


deviation  value*  (F„  lir,  =  4.83;  |i  •  .001).  and  total  system  deviation  values  (F,,.  10:,  =  5.6;  p  <  .001).  Trend 

eomparisons  of  the  transformed  values  lor  eaeli  of  these  variables  produced  significant  lim*ar  component  results  for  trials; 
no  other  orthogonal  trend  romponent  was  found  to  be  significant.  The  F  values  and  probability  levels  of  the  linear 
component  m  the  trend  analyses  are  as  follows:  dive  angle  deviation  (F|  =  1 5. 1 ;  p  <  .01 1;  airspeed  deviations  (f  117  ', 

~  1 .1.0:  p  ■  .01 );  suinmeil  longitudinal  deviation  values  (F 1  17  =  27.0;  p  <  .001 );  and  total  pickle  deviation  values 

I F 1  17  .-,  =  10.2;  p  <  .001).  These  results  combined  with  an  examination  of  the  trial  means  data  (Appendix  C.  Table 
C- 1  >  show  decreases  in  error  transformation  values  as  a  function  of  trials  for  dive  angle,  airspeed,  longitudinal,  and  total 
pickle  parameter  deviations. 

The  results  of  the  AMOY  As  for  g  and  bank  angle  transformed  values  were  not  significant,  kxaminalion  of  the 
transformed  g  mean  values  in  Appendix  C.  Table  C-l  shows  results  similar  to  those  obtained  from  analysis  of  the  30 
DRR  oil-final  g  data.  Transformed  mean  values  for  g  were  small  and  remained  consistent  over  the  training  period. 
F.xamination  of  bank  angle  transformed  mean  values  in  Appendix  C-l  also  shows  little  improvement  over  training.  During 
the  course  of  tins  effort,  there  existed  a  slight  left  to  right  drift  on  the  ASPT  v  isual  system  which  might  have  influenced 
the  bank  angle  deviation  tesults.  The  extent  to  which  this  drift  problem  confounded  the  data,  however,  was  unknown. 


2(f  DRR.  The  same  two  variable  sets  derived  from  the  .10°  DRR  data  were  used  to  examine  20°  DRR  performances. 
Similar  AMOY  As  were  computed  for  both  sets  of  variables.  The  only  significant  finding  was  for  translormed  values  of 
pickle  bank  angle  data  from  variable  set  two  (Fh,  ,  17  =  2.25;  p  <  .05).  Trend  analysis  of  these  data  revealed  a  non¬ 
significant  linear  component,  but  did  show  a  significant  quadratic  component  ( F  j  p,  -,  =  .(W;  p  <  .0251.  The  trial  means 
of  the  transformed  bank  angle  pickle  values  are  presented  in  Appendix  C.  Table  C-l.  These  results  are  not  consistent 
with  performance  trends  usually  obtained  for  training  tasks  of  this  nature.  It  is  possible  that  the  visual  drift  problem, 
identified  earlier,  may  have  influenced  training  performances  enough  to  produce  these  findings. 

Two  possible  explanations  exist  for  the  overall  inability  to  detert  performance  improvements  during  the  course  of 
training  the  20°  DRR  task.  First,  since  the  task  was  trained  immediately  following  the  50°  DRR  task,  carry-over  effects 
may  be  responsible.  Although  parameter  deviations  for  20°  bombing  events  have  greater  effect  on  bombing  impact  error 
than  they  do  for  30°  )>omhing  events,  the  20°  DRR  task  was  highly  similar  and  mechanically  was  no  more  difficult  to 
perform  than  the  30°  DRR  task.  Comparison  between  the  on-final  parameter  trial  means  for  the  two  tasks  (Appendix 
C.  Table  C-l)  shows  initial  deviation  values  for  the  20°  DRR  task  are  below  those  initiallv  obtained  for  the  30°  DRR 
task;  thus,  this  explanation  seems  plausible.  An  alternative  explanation  concerns  the  amount  of  training  devoted  to  this 
task.  Given  the  operational  constraints  imposed  upon  this  effort,  simulator  training  time  had  to  be  minimized.  Perhaps 
additional  training  in  the  simulator  would  have  resulted  in  task  performance  improvements  that  were  more  observable. 

Target  Tracking  Two  analysis  variables  were  derived  to  examine  training  performances  for  the  target  tracking 
exercise.  Both  concentrated  on  pipper  placement  error  as  the  pilot  passed  through  firing  altitude.  The  first  variable 
consisted  of  intratrial  RMS  aiming  error  in  feet.  Aiming  error  data  used  to  compute  these  values  were  collected  during 
the  period  that  the  pilot  passed  through  a  1600-foot  to  1400-foot  AGl.  firing  altitude  window.  Aiming  error  was  defined 
as  the  distance  between  the  target  and  actual  pipper  placement.  The  second  variable  was  derived  from  a  hit/miss  register 
of  pipper  placement  aeeurac  v.  A  hit  was  obtained  for  a  given  trial  whenever  the  pipper  passed  through  the  target  during 
the  firing  altitude  window  period. 

The  RY1>  aiming  error  was  analyzed  using  the  same  AMOY  A  procedure.  Results  of  this  test  were  not  significant 
for  trials.  RY1S  aiming  error  trial  means  arc  presented  in  Appendix  C.  Table  C-l.  Performance  variations  across  trials 
for  the  hit/miss  measures  were  tested  using  the  nonparametric  Cochran  Q  statistic  (Siegel,  1*4561.  Results  of  this  test 
were  significant  ((,)  -  13.55;  df  =  4:  p  <  .011.  Percentages  of  actual  hit  versus  miss  data  are  presented  by  trial  in 
Appendix  C,  Table  C-2.  These  data  show  that  the  significance  of  the  Cochran  (,)  statistic  was  due  to  improved 
performances  as  a  function  of  trials. 


-  |n  performing  this  test,  missing  observations  were  counted  as  misses.  Trial  4  contained  one  such  missing  observation,  anil  trial 
untamed  three.  This  was  deemed  the  most  conservative  approach  to  handling  the  mission  observations  problem. 


One  possible  explanation  for  the  discrepancy  between  these  test  results  is  that,  once  the  subject  placed  the  aiming 
pipper  on  the  target  at  proper  altitude,  there  was  no  real  attempt  to  hold  it  there  throughout  the  entire  firing  altitude 
period.  In  other  words,  the  subjects,  while  still  within  the  firing  altitude  window,  may  have  pulled  off  target  after  achieving 
what  they  considered  a  hit.  This  would  explain  the  increase  in  the  number  of  hits,  with  no  apparent  improvement  in 
K  MS  aiming  errors  as  a  function  of  trials. 


Aircraft  Performance  Group  Companions 

17.5  &  21 .5  Unit  A OA  Turns.  Student  mission  gradesheets  did  not  differentiate  between  17.5  and  21.5  unit  AOA 
turns:  therefore,  separate  analyses  of  gradesheet  ratings  for  the  two  events  were  not  possible.  BAM  student  performance 
questionnaires  did  differentiate  between  the  two  events  and  separate  analysis  variables  for  each  event  were  analyzed 
I  see  Appendix  (',  Table  C-3).  Better  experimental  group  performance  means  were  obtained  for  27  of  the  31  analvsis 
variables.  Results  indicated  none  of  the  differences  obtained  for  the  four  variables  showing  poorer  experimental  group 
performances  were  significant  (p  >  .05).  Of  the  27  analysis  variables  whose  group  means  indicated  better  experimental 
group  performances,  two  resulted  in  significant  findings.  These  were  (a)  V  ,,,  —  Mean  AOA  control  ratings  over  CV-5 
repetitions  for  17.5  AOA  turns  (F,  :)H  =  13.68;  p  <  .001;  U 20.20  =  298;  p  <  .01  (two-tailedl);  and  (b)  V2)  -  Mean 
AOA  control  ratings  over  CV-5  repetitions  for  21.5  AOA  turns  (F|  3#  =  5.25;  p  <  .02:  L  20  20  =  271;  p  <  .05  (two- 
tailedl).  Power  estimates  for  each  of  the  analysis  variables  are  presented  in  Appendix  C,  Table  C-3.  Aside  from  the  two 
variables  associated  with  significant  ANOVA  results  and  the  four  variables  with  negative  group  mean  differences,  power 
estimates  ranged  fromOto  .24. 

■  i0°  DRR.  Twelve  analysis  variables  were  derived  from  the  30°  DRR  data  (see  Appendix  C,  Table  C-4).  Group  mean 
differences  for  10  ol  the  analysis  variables  showed  better  experimental  group  performances  relative  to  the  control  group. 
ANOV  A  results  of  the  two  variables  showing  poorer  experimental  group  performances  were  not  significant  (p  >  .05). 
Of  the  10  other  analysis  variables  whose  group  means  indicated  better  experimental  group  performances,  one  resulted 
in  significant  findings.  This  analysis  variable  was  V2  —  number  of  repetitions  to  proficiency  |F]  ;tH  =  5.07;  p  <  .03; 
l  20,20  =  1 [i  <  .07  (two-tailed)).  Power  estimates  for  the  analysis  variables  showing  better  experimental  group 
performances,  but  which  resulted  in  nonsignificant  ANOV  As,  ranged  fromOto  .21. 

2(f  DRR.  Twelve  analysis  variables,  similar  to  those  derived  from  the  30°  DRR  data,  were  derived  from  the  20° 
DRR  data  (see  Appendix  (',  Table  C-5).  Croup  mean  differences  for  eight  of  the  analysis  variables  showed  better 
experimental  group  performances  relative  to  the  control  group.  One  analysis  variable  showed  no  difference  between 
groups  lV’2  —  number  of  repetitions  to  proficiency).  ANOVA  results  for  the  three  variables  which  showed  better  control 
group  performance  were  not  significant  Ip  >  .05).  Test  results  of  all  other  20°  DRR  variables  found  one  to  be  significant. 
This  finding  came  from  analysis  of  variable  V(,  —  mean  of  the  overall  repetitions  ratings  on  C\  -5  repetitions  (F| ..m  = 
4.29:  p  <  .04:  l  20.20  =  269;  p  <  .06:  (two-tailedl).  Power  estimates  for  those  non-significant  differences  showing  better 
experimental  group  performances  ranged  from  Oto  .24. 

Target  Tracking.  Five  analysis  variables  were  derived  from  the  aircraft  target  tracking  training  data  (see  Appendix 
(\  Table  G-6).  Group  mean  differences  for  all  variables  showed  better  experimental  group  performances  relative  to  the 
control  group.  Although  none  of  the  variables  produced  significant  A  NOV  A  results  at  the  p  =  .05  level,  two  approached 
significance.  One  of  these  two  variables  produced  significant  Mann- V)  hitnev  l  results.  These  two  variables  were  lal  V  \ 

—  Mean  of  the  Overall  Repetition  Ratings  (F|  ;tH  =  3. 13;  p  <  .09:  l  20.20  =  260;  p  <  .  10  (two-tailedl);  and  (bl  V  , 

-  Gradeslip  Ratings  (F,  )B  =  3.3;  p  <  .08:  U  20.20  =  266;  p  <  .03  (two-tailed)l.  Power  estimates  over  all  variables 
ranged  from  Oto  .32. 

The  evidence  for  training  transfer  of  the  target  tracking  task  seems  stronger  than  that  obtained  for  the  three  previous 
exercises.  IX  ith  the  number  of  analyses  conducted  for  each  of  the  experimental  tasks,  however,  conclusions  should  be 
based  on  trends  in  the  results  rather  than  01.  individual  findings  of  a  few  analyses  which  reach  significant  levels  of 
probability.  This  issue  is  presented  in  greater  detail  in  the  discussion  section. 
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IV.  PH  ASH  II -1.1 -N 


Subject* 

The  subject*  lor  I’lmse  II  were  ,'W>  A- 10  B-courae  student  pilots  assigned  to  the  355th  Tactical  Training  ing.  Davis- 
Vlonthun  AKB.  All  students  had  participated  as  either  an  experimental  or  a  control  group  subject  in  Phase  I  of  this  effort. 
Kaeh  subject  was  a  participant  in  the  A- 10  ASPT  surface  attack  phase  training  program.  Student  pilots  from  Phase  I 
control  and  experimental  groups  were  assigned  to  one  of  two  groups.  Although  squadron  scheduling  conflicts  prohibited 
the  use  of  strict  counterbalancing  procedures,  each  Phase  II  group  was  composed  of  an  approximately  equal  number 
of  subjects  from  each  of  the  two  Phase  I  groups.  Sample  size  differences  resulted  due  to  the  scheduling  conflicts  and 
ASP!’  time  constraints.  Fifteen  subjects  were  assigned  to  the  Phase  II  experimental  group.  The  remaining  21  subjects 
served  as  controls.  Time  constraints  also  prohibited  a  control  sortie  being  given  to  control  group  subjects  as  was  done 
in  Phase  I. 


Kxperiinental  T  raining 

I.I.N  simulator  profiles  were  develo|wd  with  the  help  of  IPs  from  the  A- 10  operational  training  development  section 
at  Davis-Monthan  AFB.  Upon  completion  of  ASPT  surface  attack  training,  experimental  group  subjects  received  a  l,l,N 
briefing  whic  h  covered  the  visual  environment,  task  requirements,  maps  of  the  I.I.N  routes,  and  procedures  for  the 
experimental  sortie.  After  receiving  this  briefing,  subjects  flew  two  I.LN  routes  on  the  ASPT  with  an  A-10  IP  providing 
instruction  from  the  main  console. 

Limitations  in  the  image  generation  capabilities  of  the  ASPT  prohibited  development  of  a  highly  detailed  visual 
environment.  In  addition,  development  of  a  visual  environment  which  simulated  the  actual  routes  used  for  training  in 
the  aircraft  was  prohibited  due  to  time  and  resource  constraints.  The  LI.N  simulator  profiles  therefore  emphasized 
procedural  training  of  tasks  relevant  to  I.I.N  operations.  The  visual  environment  used  in  these  profiles  was  selected  on 
the  basis  of  area  size  and  visual  cue  requirements  of  the  I.I.N  exercise.  The  visual  environment  selected  was  an  accurately 
scaled  representation  of  the  Fulda,  West  Germany  area.  The  main  area  of  use  was  approximately  150  miles  *  1 50  miles. 
Actual  maps  of  the  Fulda,  West  Germany,  area  were  used  to  place  mountains,  roads,  towns,  and  other  major  landmarks 
throughout  the  simulated  environment.  F.levations  deviated  from  those  indicated  on  the  real  maps  of  the  area,  but 
distances  between  landmarks  were  to  scale.  The  ground  level  was  1000  feet  above  mean  sea  level  (MSI.)  throughout 
the  simulated  environment  except  for  mountain  terrain  areas. 

A  highly  detailed  runway  wus  developed  at  Fulda  and  used  as  home  base.  Two  route  entry  points  were  positioned 
close  to  this  home  base.  F  rom  each  enlrv  (mint,  low  level  corridors  were  constructed  extending  into  the  target  area.  One 
of  these  corridors  consisted  of  three  legs;  the  other  corridor  consisted  of  four  legs.  Routes  1  and  2  were  mirror  images 
of  each  other;  i.e. .  ingress  leg-  for  route  1  served  as  egress  legs  for  route  2  and  vice  versa.  The  average  length  of  each 
leg  was  about  25  miles.  Numerous  trees,  buildings,  grain  silos,  and  towns  were  used  for  low  altitude  visual  cues  within 
the  low  level  corridors.  The  density  of  these  cues  was  satisfactory  for  sustained  flight  at  or  above  300  feet  AGL.  The 
v  isual  cues  were  scattered  within  2  miles  either  side  of  the  centerline  of  each  leg,  giv  ing  the  corridors  a  width  of 
approximately  V  miles.  Towns,  factories,  road  intersections,  lowers,  and  mountains  were  used  as  checkpoints. 
Ke|>rcscntatinns  of  tanks  were  used  as  targets  in  the  target  area.  The  total  flying  distance  from  home  base  to  the  target 
area  was  almiit  I  10  miles  on  either  corridor. 

Students  navigated  the  routes  using  tojtographic  maps  detailing  the  CGI  env  ironment  displayed  on  the  ASPT  visual 
system.  The  maps  were  color  copier  reproductions  of  photographic  maps  detailing  the  low  level  corridors.  The 
jihotograjiluc  maps  were  developed  from  an  original  artwork  majiping  all  landmarks  generated  within  the  environment. 
Care  was  taken  to  obtain  c  orrect  coloring  and  scaling  of  the  final  rnap  reproductions.  The  maps  were  bound  together 
lo  form  separate  low  level  ma|>  Issiklets  of  eac  h  route  with  parameter  information  displayed  in  a  format  identical  to  what 
I-  use  1  ii  airc  raft  I  I  N  operations. 


Karl)  exercise  began  willi  u  takeoff  from  the  Fulda  airyiort.  The  student  was  vectored  to  the  LL.N  route  entry  point 
that  marked  the  beginning  of  the  navigation  exerc  ise.  The  student  flew  the  low  level  route  to  the  target  area,  where  strafe 
lu-k-  were*  performed  using  tanks  as  targets,  (.poll  completion  of  the  strafe  tasks,  the  student  flew  to  the  egress  entry' 
point  and  followed  a  low  level  route  hac  k  to  Fulda.  The  only  real  differences  between  the  two  routes  were  the  leg  headings 
down  to  navigate  the  routes  and  the  strafe  approaches  performed  in  the  target  area  (on  route  1.  subjects  performed 
conventional  approaches  to  strafe;  on  route  2.  subjects  performed  pop-up  approaches!.  Detailed  syllabi  of  the  two  sorties 
are  provided  in  Table  f. 


Table 4.  Detailed  LLN  ASPT  Syllabus 


Route  1 

1 .  Takeoff;  Climb;  level  off  (u  6000 feet  AGl.fWX:  27, (XX) feet;  Winds;  25  knots  gusting  to  50 knots  (3-  560°) 

2.  Vector  to  Checkpoint  A 

5.  l.I.N  Ingress:  I  Altitude  -  500  foot  AGI.:  Airspeed  -  .100  knots) 

a.  Checkpoint  A;  Intersection  ofThree  Roads  (Heading  -  091  °| 

b.  Checkpoint  B:  Railroad  Depot  (Heading  -  151°) 

c.  Check[)«int  C:  Town  W  (Heading -059) 

d.  (.heck [Hunt  I):  Factory;  Target  Identification  Point 

f.  long  Range  Strafe  (Conventional  -5  repetitions) 

5.  Two  Target  Strafe  (Conventional  -  5  repetitions) 

6.  l  .I.N  F.gress:  (  Altitude  -  500  feet  AGI.;  Airspeed  -  500  knotsl 

a.  Checkpoint!):  Factory  (Heading  -  224°) 

b.  Checkpoint  E:  Town  W  (Heading  -274°) 

e.  Checkpoint  F:  Town  X  (Heading- 222°) 

d.  CheckpointG:  Town  Y  (Heading-511°t 

e.  Checkpoint  H:  Town/ 

7.  Return  to  Fulda  A  AF 


Route  2 

1 .  Takeoff;  Climb;  level  off  (<t  6000 (eel  AGIdWX:  27,000 feet;  Winds;  25  knots  gusting  to .50  knots  (it  560°) 

2.  \  eetor  to  Checkpoint  A 

5.  l.I.N  Ingress:  (  Altitude-  500 feet  AGI.;  Airs|ieed  -  500 knotsl 

a.  < Checkpoint  A:  5 own /. (Heading -  1  1 7°) 
h.  Checkpoint  II:  5  own)  l  Heading -059°) 

c.  Checkpoint  C:  Town  X  (Heading -094°) 

d.  Checkpoint  I);  Town  W  (Heading -042°) 

e.  Checkpoint  K:  Factory:  Target  Identification  Point 

f.  I  ving  Range  Strafe  (Pop-up  5  repetitions) 

5.  Two  Target  Strafe  ( Pop-up  -  5  rejietitioiis) 

6.  1.1  N  F.gress.  I  Altitude  -500  feet  AGI.;  Airspeed  -  500  knots) 

a.  Checkpoint  F:  Factory  (Heading  -  224°) 

b.  Checkpoint  F:  Town  W  (Heading -  552°) 

c.  CheckpointG:  Railroad  Depot  I  Heading  269°) 
d  Checkpoint  H:  Intersection  of  Three  Roails 


7.  Return  to  Fulda  AAF 


Kor  Inith  routes,  winds  were  set  at  20  knots,  gu  sting  to  .'10  knots,  with  delays  of  15  to  .10  seconds  between  gusts. 
A  5-second  buildup  time  to  maximum  gust  was  programmed,  and  the  duration  of  the  gusts  varied  from  15  to  30  seconds. 
Wind  direction  varied  ±  lOdegrees  from  true  north.  Direction  variations  were  made,  on  average,  every  20  seconds. 
Turbulence  was  simulated  at  a  moderate  level  and  visibility  was  set  at  4  1/2  miles. 


Dependent  Measures 

ASPT  Performance  Measurement.  Difficulties  with  the  software  logic  used  to  capture  ASPT  performance  measures 
were  encountered  throughout  Phase  II.  This  problem  made  questionable  the  validity  of  all  ASPT  LLN  performance  data, 
and  analyses  of  these  data  are  not  reported.  The  measurement  difficulties  did  not  affect  the  quality  of  the  actual  LLN 
simulation  flown  by  subjects.  All  experimental  group  subjects  flew  the  entire  LLN  simulator  profile.  The  training  transfer 
issue  for  this  exercise  was  addressed  solely  by  between  group  aircraft  performance  analyses. 

Aircraft  Performance  Measurement.  Student  data  on  LLN  aircraft  performances  were  obtained  using  procedures 
similar  to  those  used  for  collecting  BAM  aircraft  performance  data  in  Phase  I  of  this  effort.  The  procedure  incorporated 
l«>th  LLN  performance  questionnaires  developed  for  this  effort  and  student  gradesheets.  Both  measurement  instruments 
were  completed  by  flight  IPs  at  the  debriefing  of  each  flight.  All  LLN  missions  were  flown  with  the  student  flying  lead 
and  the  IP  flying  chase.  As  in  Phase  I,  IP/student  assignments  were  usually  based  on  IP  availability  and  student  progress. 
Lach  student  was  scheduled  to  fly  two  aircraft  sorties  containing  LLN  exercises  (BAM-1  and  SA-1).  Not  all  subjects, 
however,  performed  the  scheduled  number  of  LLN  missions.  Three  subjects  from  the  experimental  group  and  four 
subjects  from  the  control  group  had  missing  data  for  one  LLN  exercise  due  to  weather  restrictions  on  the  day  of  flight. 
Complete  sets  of  LLN  aircraft  performance  data  were  collected  for  the  remaining  29  subjects.  All  data  obtained  were 
used  in  the  final  analyses. 

The  student  mission  gradesheets  used  for  LLN  missions  differed  from  the  conversion  phase  gradesheets  only  on 
the  tasks  listed  for  rating.  The  procedures  and  the  rating  scale  used  were  identical.  IPs  also  completed  the  LLN  student 
performance  questionnaire  presented  in  Appendix  B.  Table  B-2  for  each  exercise  performed.  Using  this  questionnaire, 
IPs  provided  the  following  data  for  each  leg  of  the  exercise:  (a)  number  of  times  lost  or  disoriented,  (bl  time  deviation 
on  checkpoint,  (c)  average  altitude  deviation,  (d)  average  airspeed  deviation,  and  (e)  rating  on  course  control.  Based 
on  IP  interviews  and  review  of  mission  objectives,  these  measures  were  determined  to  reflect  the  most  important 
performance  aspects  of  the  LLN  mission.  The  same  procedures  used  to  brief  IPs  on  experiment  requirements  in  Phase 
I  of  this  experiment  were  also  used  in  the  LLN  phase.  Likewise,  a  researcher  was  on  site  during  the  data  collection  period 
to  administer  questionnaires,  to  collect  completed  measurement  materials,  and  to  answer  questions  pertaining  to  the 
effort. 


Data  .Analysis  Procedures 

As  previously  noted,  due  to  questionable  validity,  analyses  of  the  LLN  ASPT  performance  data  are  not  reported. 
Aircraft  performance  data  were  auulyzed  in  the  following  manner.  The  seven  analysis  variables  presented  in  Appendixes 
D,  Tables  11-1  and  11-2  were  derived  for  each  of  the  two  LLN  sorties.  Selection  of  analvsis  variables  was  based  on  a 
desire  to  derive  variables  most  reflective  of  mission  performance  and  to  achieve  compatibility  with  simulator  performance 
analvsis  variables  planned  for  in  the  original  design,  for  Five  of  these  variables,  means  were  computed  over  data  obtained 
for  each  route  leg  in  order  to  make  compatible  data  from  different  routes  with  varying  number  of  legs.  Single  factor 
ANOVA  and  unequal  sample  size  procedures  (Keppel,  I97d)  were  used  to  lest  between  group  differences  for  each 
variable.  Post  hoc  determination  of  statistical  power  (Keppel,  l(J7dl  was  also  computed  for  each  variable  to  assess  the 
sensitivity  of  thcdat.i  to  group  j>erformanee  variations. 


Phase  1 1  Results 


(»ronp  mean  differences  for  three  of  the  seven  BAM- 1  sortie  analysis  variables  showed  letter  experimental  group 
performances  relative  to  the  control  group  (see  Appendix  1).  Tabic  l)-l ).  None  of  these  differences  were  significant  Ip 


>  .05).  Four  variables  showed  lietter  control  group  performances.  Statistical  analysis  of  these  data  found  group 
differences  for  mean  airspeed  deviations  (Vs)  to  be  significant  (Fi_.ii  =  10.25;  p  <  .01 );  the  results  of  the  analyses  for 
the  remaining  three  variables  were  not  significant  (p  >  .05).  ('.roup  means  for  all  SA-1  sortie  variables  indicated  better 
control  group  performances  (see  Appendix  D,  Table  D-2).  Results  of  the  statistical  analyses  of  these  data  found  none 
ul  these  differences  to  lie  statistically  significant. 


V.  DISCI  SSION 


Significant  task  improvements  over  BAM  simulator  periods  were  demonstrated  for  three  of  the  five  tasks  trained. 
Analyses  of  the  aircraft  performance  data  for  these  three  tasks  resulted  in  significant  positive  findings  for  one  of  the  17 
analyses  computed  for  the  17.5  AOA  turns  task,  one  of  the  12  analyses  computed  for  the  50°  DRR  task,  and  two  of 
the  five  analyses  computed  for  the  target  tracking  task.  Similar  results  were  obtained  from  aircraft  performance  analyses 
of  the  remaining  BAM  tasks:  one  of  17  analyses  for  21.5  AOA  turns  data  was  significant,  and  one  of  12  analyses  for 
20°  DRR  data  was  significant.  Although  few  of  the  aircraft  data  analyses  resuited  in  significant  findings,  group  means 
obtained  for  these  data  showed  an  overall  trend  indicative  of  better  performance  by  the  simulator  trained  group.  Simulator 
task  performances  could  not  be  examined  over  the  course  of  1,1. N  training  due  to  apparatus  failure.  Analyses  of  the  LLfV 
aircraft  performance  variables  were  not  indicative  of  positive  simulator  to  aircraft  transfer  of  training. 

Two  factors  which  may  have  influenced  the  results  of  this  effort  are  presented  for  consideration.  The  first  factor 
concerns  the  adequacy  of  the  training  procedures.  There  are  three  components  to  this  factor.  The  first  component  concerns 
the  fact  that  the  simulator  scenarios  for  all  tasks  were,  by  necessity,  restricted.  The  scope  of  the  effort  did  not  allow 
the  determination  of  performance  ceiling  effects  for  task  training  in  the  simulator.  Even  for  BAM  tasks  whose  simulator 
performance  data  showed  significant  improvement  as  a  function  of  training,  the  effect  of  additional  training  in  the 
simulator  on  either  simulator  or  aircraft  task  performances  is  unknown.  The  second  component  concerns  the  period 
between  training  in  the  simulator  and  aircraft  testing.  This  period  was  not  immediate:  approximately  1  week  elapsed 
between  BAM  training  in  the  ASPT  and  the  first  BAM  aircraft  sortie:  5  to  5  days  typically  elapsed  between  the  ASPT 
1.1. N  sorties  and  the  first  l.LN  aircraft  sortie.  The  third  component  is  concerned  primarily  with  LLN  training  operations. 
The  ASl’T's  l.l.N  visual  environment  was  rather  impoverished  due  to  limitations  of  the  ASPT  CGI  capability.  None  of 
the  experimental  pilots  became  lost  during  the  course  of  performing  LLN  simulator  scenarios.  Additionally,  the  ASPT 
l.LN  visual  environment  was  a  representation  of  an  area  dissimilar  to  the  real  environment  used  for  training  in  the  aircraft. 
As  a  result,  the  ASPT  LI.N  scenarios  provided  what  might  lie  termed  generic  procedural  training;  by  no  means  were 
they  intended  to  provide  visual  familarization  of  the  test  environment  used  for  training  in  the  aircraft.  Although  these 
issues  may  have  negatively  influenced  the  results  of  this  effort,  lack  of  complete  control  over  training  operations  is 
eommonplace  in  operational  settings.  The  effort  was  designed  to  provide  results  based  on  antieipated  usage  of  sueh 
training  programs  by  the  operational  command.  As  such,  the  effort  succeeded  in  arromplishing  this  objective  while  still 
maintaining  the  control  over  experimental  procedures  required  by  the  methodology  and  analysis  procedures  used. 

The  second  factor  to  be  considered  involves  the  sensitivity  of  the  dependent  variables  used  to  deteet  aircraft  task 
performance  variations.  Previous  ASPT  experiments  involving  weapons  delivery  training  have  demonstrated  significant 
transfer  to  the  aircraft  when  the  dependent  variables  are  based  on  objective  measures  of  aircraft  performance.  Gray  et 
al.  1 1981)  used  bombing  error  and  percent  strafe  hits  to  demonstrate  that,  for  five  of  six  weapons  delivery  tasks  tested, 
A-I0  B-course  pilots  trained  on  the  ASPT  performed  significantly  better  on  initial  A-10  surface  attack  aircraft  sorties 
than  did  control  counterparts.  Similar  results  were  obtained  by  Gray  and  Fuller  1 1977).  In  this  experiment,  fighter  lead-in 
student  pilots  were  trained  on  weafions  delivery  tasks  in  the  ASPT  and  tested  against  control  pilots  in  the  F-5B  aircraft. 
Hie  aircraft  performance  variables  used  in  this  effort  were  derived  from  bombing  error  and  IP  evaluations  of  performances. 
Bombing  error  variables  showed  significantly  better  experimental  group  performances  for  all  weapons  delivery  tasks 
tested;  however,  none  of  the  groiqi  differences  obtained  for  the  IP  evaluation  variables  were  significant.  As  target  tracking, 
20°  DRR  and  50°  DRR  tasks  are  all  similar  in  nature  to  their  corresponding  weapons  delivery  tasks,  these  previous  efforts 
suggest  that  stronger  evidence  supporting  transfer  of  training  may  have  resulted  if  more  objective  aircraft  performances 
metrics  were  available.  The  measurement  sensitivity  is  sue  cannot  be  conclusively  resolved  here;  however,  power  values 
determined  for  aircraft  performance  data  analyses  and  inferences  made  on  the  basis  of  results  from  previous  efforts  make 
questionable  the  use  of  conventional  IP  evaluation  procedures  for  test  and  evaluation  purposes. 
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IP  evaluations  of  student  performances  have  long  been  used  by  TAC  in  determining  individual  pilot  proficiency 
and.  in  geneal,  the  Command’s  overall  combat  readiness.  The  standardization/evaluation  (STAN/EVAL)  program  serves 
the  function  of  providing  commanders  with  meaningful  indicators  that  reflect  aircrew  training  and  capability  to  perform 
the  unit  mission.  To  achieve  these  ends,  the  STAN/EV  AL  program  attempts  to  control  the  evaluation  process  and  ensure 
standardization  among  the  training  units.  In  addition,  criteria  for  IP  assignment  of  student  performance  ratings  are 
specified  by  criterion  referenced  objectives  (CROs).  One  inherent  problem  is  that,  even  by  specifying  the  rating  criteria, 
this  evaluation  process  has  no  provision  for  verifying  the  subjective  values  produced.  Additionally,  the  effect  that 
confounding  factors  have  on  the  final  evaluations  is  difficult  to  ascertain.  A  student's  previous  performance  history, 
attitude,  and  intentions  may  all  significantly  influence  the  subjective  performance  values  assigned.  As  part  of  the  STAN/ 
EVAI.  program,  students  are  measured  against  expected  competence  levels  annotated  in  course  syllabi.  To  this  extent, 
the  STAN/EV  Al.  program  itself  may  influence  IP  evaluations  with  factors  unrelated  to  student  performance. 

On-going  and  planned  procurements  of  new  weapon  system  operational  flight  trainers  (OFTs)  for  TAC  pilot  training 
have  highlighted  the  need  for  evaluating  these  trainers  in  terms  of  their  contribution  to  flying  training  programs.  Test 
and  evaluations  are  planned  for  these  trainers  using  methodologies  and  subjective  performance  metrics  similar  to  those 
used  in  the  current  effort.  Examination  of  the  power  estimates  presented  in  the  appendixes  provide  insight  to  the  kind 
of  results  to  be  expected  unless  another  approach  is  taken.  Power  estimates  can  be  interpreted  in  the  following  manner. 
Given  the  sample  size  and  variability  of  the  data  between  and  within  groups  for  a  given  measure,  the  probability  of 
obtaining  significant  differences  between  groups  is  equal  to  the  obtained  power  value.  The  power  estimates  presented 
in  the  appendixes  were  derived  from  mean  squared  data  from  the  computed  ANOVAs  and  used  the  F  distribution  in 
determining  the  power  values.  As  a  result,  the  power  estimates  are  conservative,  by  nature  of  the  F  distribution.  Even 
when  more  liberal  methods  are  used,  however,  the  resulting  power  estimates  are  quite  moderate.  Power  increases  as 
sample  sizes  increase,  between  group  variability  increases  or  the  within  group  variability  decreases.  Large  sample  sizes 
are  difficult  to  obtain  for  operational  testing  purposes.  A  desirable  alternative  to  increasing  sample  sizes  for  purposes 
of  increasing  jxiwer  would  be  to  use  measurement  techniques  that  minimize  sources  of  unwanted  variability.  The 
feasibility  of  adapting  alternative  measurement  systems  for  aircraft  performance  assessments  needs  to  be  evaluated  and, 
if  suitable,  incor|x>rated  into  future  simulator  test  and  evaluation  programs. 


VI.  CONCLUSIONS  AND  RECOMMENDATIONS 

Two  factors  have  been  considered  that  may  have  negatively  affected  the  outcome  of  this  effort.  First,  training  in 
the  simulator  was  too  restricted  to  ensure  adequate  transfer  effects.  Second,  conventional  IP  evaluation  procedures  were 
not  sufficiently  sensitive  for  detecting  group  performance  differences  in  the  aircraft.  Whether  the  obtained  results  were 
due  to  one,  neither,  or  a  combination  of  these  proposals  is  unknown.  The  implications  of  these  results  form  the  basis 
of  the  following  recommendations  for  both  current  and  future  training  evaluation  programs. 

1.  Ensure  the  optimal  amount  of  benefit  is  derived  from  the  similator  if  transfer  of  training  is  an  issue.  This  can 
be  accomplished  by  pretesting  pilots  in  the  simulator  for  performance  ceiling  effects,  then  programming  enough  simulator 
time  to  train  pilots  to  optimal  proficiency  during  the  test  period. 

2.  Evaluate  the  metrics  used  to  assess  task  performances  in  terms  of  suitability  and  possible  effects  of  unwanted 
sources  of  variance.  When  available,  use  objective  performance  measures.  If  subjective  performance  evaluations  are 
necessary,  rater  instruction  sessions  and  pretest  procedures  should  be  used  to  help  ensure  interrater  reliability. 
Minimizing  the  number  of  raters  used  in  (he  test  and  counterbalancing  raters  with  ratees  are  also  recommended. 


t 
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Table  A-l.  A- 10  ASPT  Detailed  Conversion  Pha»e  Syllobiu 


AS- 1 

Simulalor 

1.  Takeoff;  Climb;  Level  Off 

2.  Stall  Recovery  Procedures  (Familiarization  Only) 

a.  Vertical  Confidence  Maneuver 

b.  Power  on  Stalls  (Straight  Ahead  and  Turning) 

c.  Traffic  Pattern  Stall  Series 

d.  Slow  Flight 

d.  Lazy  F.ight;  Aileron  Rolls  (Speed  Brakes  Closed  and  Speed  Brakes  Open  -  Familiarization  Only) 
V.  Osip,  Cuban  F.ight;  Split  S 

5.  Descent.  Straight-ln  Approach;  Go  Around 

6.  Re-F.ntrv;  Normal  Overhead  Pattern  and  Approach;  Go  Around 

7.  Closed  Pattern:  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

8.  Closed  Pattern;  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

0.  Closed  Pattern;  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

10.  Closed  Pattern;  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

I  I .  (dosed  Pattern;  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

12.  Re-Kntrv;  Normal  Overhead  Pattern  and  Approach;  Full  Stop  Landing 

AS-2 

Simulator 

1 .  Takeoff:  Climb;  Level  Off 

2.  Steep  Turns 

.1.  Re-Fntry;  Straight-ln  Approach  (GCA);  Go  Around 

4.  (dosed  Pattern;  Normal  Approach;  Touch  and  Go  (Familiarization  Only) 

5.  (dosed  Pattern;  No  Flap  Approach;  Touch  and  Go  (Familiarization  Only) 

6.  Closed  Pattern;  No  Flap  Approach;  Touch  and  (^(Familiarization  Only) 

7.  Straight-In  Approach  (Simulated  Single  Engine);  Simulated  Single  Engine  Go  Around 

8.  Straight-ln  Approach  (Simulated  Single  F.ngine);Touch  and  Go  (Familiarization  Only) 

9.  (dosed  Pattern;  Normal  Approach;  Full  Stop  landing 

10.  Takeoff  (dOl;  Departure 

1  1 .  .(()  Re-Entry 


1:1  Ratio 
1.5  Hours 


1:1  Ratio 
1.5  Hours 
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Table  A-2.  A- 10  ASPT  Detailed  Surface  Attack  Syllabus 


\S-3 

Simulator 

1.  Takeoff;  Climb;  1  evel  Off 

2.  Range  Orientation  and  Dive  Bomb  Demonstration  (Familiarization  Only) 

3.  30°  Dive  Bomb  (HOT)  -  3  repetitions 

l.  45°  High  Angle  Dive  Bomb  (HOT)  -3  repetitions 
3.  low  Angle  Low  Drag  (HOT) -3  repetitions 
ti.  low  Angle  Bomb  (HOT)  -  3  repetitions 

7.  low  Angle  Strafe  (HOT)  -  5  repetitions 

8.  Kange  Departure 

9.  Hung  Ordnance  Pattem(Runway  12);  Full  Stop  landing 

AS-4 

Simulator 

1.  Takeoff;  Climb,  Level  Off 

2.  Range  F.ntry 

3.  30°  Dive  Bomb  (HOT) -3  repetitions 

4.  45°  High  Angle  Dive  Bomb  (HOT) —  3  repetitions 
3.  low  Angle  Low  Drag  (HOT)  -  3  repetitions 

(>.  low  Angle  Bomb  ( HO'r)  -  3  repetitions 

7.  low  Angle  Strafe  (HOT)  -  5  repetitions 

8.  High  Angle  Strafe  (HOT)  -  5  repetitions 

9.  Range  Departure 

10.  Hung  Ordnance  Pattern  (Runway  30);  Full  Stop  Landing 

AS- 5 
Simulator 

1.  Takeoff;  Climb;  Level  Off 

2.  Range  F.ntry 

3.  low  Angle  Bomb  Curvilinear  (HOT) -2  repetitions 

4.  low  Angle  Bomb  Pop-Up  Demonstration  -  1  repetition  (Familiarization  Only) 

5.  low  Angle  Bomb  Pop-Up(DRY)-  1  repetition 
(>.  low  Angle  Bomb  Pop-Up(HOT) -6  repetitions 

7.  low  Angle  Strafe  Pop-Up  Demonstration  -  1  repetition  (Familiarization  Only) 

8.  low  Angle  Strafe  Pop-Up  (DRY)  -  1  repetition 

9.  low  Angle  Strafe  Pop-Up  (HOT)  -  5  repetitions 

10.  Range  Departure 

I  1 .  Normal  Overhead  Pattern  and  Approach;  Full  Stop  Landing 


1:1  Ratio 
1.5  Hours 


l:lRatio 
1.5  Hours 


1:1  Ratio 
1.5  Hours 
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APPENDIX  B:  AIRCRAFT  STUDENT  PERFORMANCE  QUESTIONNAIRES 
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Table  H-l  .  Elementary  BAM  Evaluation  Form 

(to  be  completed  for  sorties  CV4  through  CV6I 


Squadron 


Date 


Mission 


Pilot  Name 


IP  Name 


Event:  Results  by  Trial 

~  :  i  2  3 

1 .  30°  Dive,  Roll  and  Recovery 

A/S  Deviation  (a.  2500  ft  (—  or -)  _ _ _ _ 

Dive  Anglo  Deviation  (w  2500ft  (—  or  — )  _ _ . _ _ _ 

Overall  Rating (0-4)  _  _  _ 

2.  20°  Dive,  Roll  and  Recovery 

A/S  Deviation  («'  1 800  ft  ( —  or  —  >  _  _  _ 

Dive  Angle  Deviation  (5)  1800ft  (-or  — )  _  _  _ — . — - 

Overall  Rating(0-4|  _  _ _  _ 

3.  Target  Tracking 

Overall  Rating  _  _  — - - 

1.  17.5  AOA  Turns  (90/180) 

Vlax  Altitude  Deviation  (-or-)  _ _  _ 

AOA  Control  (0-4  Rating)  _ _ _ .  _  _ 

Overall  Rating (0-4  Rating)  _ _  _ 

5.  2 1.5  AO  A  Turns  (90/ 180) 

Max  Altitude  Deviation  (- or -)  _  _  — - 

AOA  Control  (0-  V  Rating)  _  _ _ — - - 

Overall  RatinglO -4  Rating)  _____  - - — - 


» 
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Table  B-2.  A.  10  Low  Level  Navigation  Evaluation  Form 

(to  be  completed  at  debriefing  for  sorties  BAM- 1  and  SA-1 ) 


Sfjtjwlnxt  _ _ _ _ _ _ Date _ _ _ _ Mission  & _ 

Pilot  Name _ _  IF  Name_ _ 

Route  # _ Entry  Point _ Exit  Point. 

Leg:  1  2 

1.  Number  of  Times  Lost  (disoriented)  _ 

2.  Time  Deviation  on  Checkpoint  (Secs  -  or  -)  _ 

3.  Average  Altitude  Deviation  (-  or  -)  _ 

4.  Maximum  Airspeed  Deviation  (-  or  -)  _ 


5.  Course  Control  (0-4  Rating) 


APPENDIX  C.  PHASE  I  TASK  VARIABLES,  GROUP  MEANS  AND 
AIRCRAFT  DATA  F  VALUES  WITH  STATISTICAL  POWER  ESTIMATES 
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Table  Cl.  ASPT  BAM  Variable  Trial  Mean* 


Trial  Mean* 


Maneuver 

Variable 

1 

2 

3 

4 

3 

6 

7 

8 

V 

to 

.  1  7.5  l  nit 

RMS  Altitude 

AOA  Turns 

Deviation 

228.2 

214.9 

195.8 

183.0 

180.3 

205.0 

113.2 

152.0 

143.7 

108.7 

21.51  nil 

RMS  Altitude 

AOA  Turns 

Deviation 

11)7.7 

150.0 

165.4 

144.9 

131.3 

136.0 

111.6 

107.9 

131.4 

HO  1 

t.  :m)°drr 

On  Final  Parameter*: 

RMS  Dive  Angle 
Deviation 

R MSG  Values 

3.76 

4.02 

3.20 

2.80 

3.00 

2.34 

2.20 

(desired:  .88) 

1.12 

1.06 

1.00 

1.05 

1.06 

1.05 

1.01 

Pickle  Parameters: 

Dive  Angle 

Deviation 

108.9 

94.5 

64.4 

69.7 

72.9 

48.0 

41.4 

Airspeed 

Deviation 

G- Value 

81.0 

85.7 

62.4 

57.3 

58.4 

61.4 

59.4 

Deviation 

49.4 

46.1 

49.0 

55.7 

43.3 

38.6 

35.2 

longitudinal 
Total  Error 
Bank  Angle 

239.  .5 

226.3 

175.8 

182.7 

174.6 

148.0 

136.0 

Deviation 

Total  System 

63.0 

73.0 

53.4 

57.4 

44.2 

36.1 

46.7 

Error 

302.3 

299.3 

229.2 

240.1 

218.8 

184.1 

182.7 

20°  DRR 

On  Final  Parameters: 

RMS  Dive  Angle 
Deviation 

RMS  G  Values 

2.81 

2.47 

2.23 

2  88 

2.76 

2.12 

2.44 

(desired:  .94) 

1.13 

1.12 

1.24 

1.13 

1.14 

1.12 

1.06 

Pickle  Parameters: 

Dive  Angle 
Deviation 
Airspeed 

81.4 

72.6 

43.0 

79.9 

84.1 

57.2 

76.5 

Deviation 

G  V  alue 

90.0 

93.4 

68.6 

67.9 

55.9 

79.7 

79.5 

Deviation 

42.0 

45.5 

66.8 

76.8 

79.3 

41.1 

64.8 

longitudinal 
Total  Error 
Bank  Angle 

213.4 

211.5 

178.4 

224.6 

219.3 

178.0 

220.8 

Deviation 

44.7 

44.2 

74.0 

79.1 

54.4 

49.5 

36.7 

Total  System 

Error 

258.1 

255.7 

254.4 

303.7 

273.7 

227.5 

257.5 

i.  Target 

RMS  Aiming 

Tracking 

Error 

135.1 

153.4 

144.8 

215.0 

131.6 

Table  C-2.  Percentage  of  Hit*  by  Trial  for  Target  TraclungTaak 


Trial 

1 

2 

3 

4 

s 

Hits,  Percent 

Number  of 

.45 

.55 

.64 

.76 

.84 

Observations* 

22 

22 

22 

21 

19 

‘Fluctuations  in  numbers  of  observations  are  due  to  performance 
measurement  system  failures. 
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Tahir  ('.  i.  Vnrialili'K,  Oritup  Iran* inJ  Sutntical  Power  Kilimalrt  { alpha  =  .OS) 

for  Aircraft  Turns  Data 


Variable 

Mew 

e*p 

F  Value 
<df  1.38) 

Power 

Kattakale 

V  ,  -  Highest  Re|ielilioii  Haling 
obtained  in  MV,  1 17.. 5  Turns) 

3.1 

3.05 

.20 

* 

V'a-Hijshcsl  Repetition  Rating 
obtained  in  A/C(21.5Tums) 

3.25 

3.05 

2.27 

.21 

V  Number  of  Repetitions  to 

Profirienev  1 1 7.5Turns) 

1.45 

1.1 

3.65 

** 

V  v-  Number  of  Repetitions  to 

Proficiency  (21 .5  Turns) 

1.35 

1.45 

.15 

* 

V  1"  Repetition  Overall 

Rating  on  CV-4  ( 1 7.5  Turns) 

2.0 

2.06 

.23 

** 

V(l-  Mean  of  Altitude  Deviations 
on  CV-4  Repetitions  ( 17. 5  Turns) 

108.04 

123.14 

.50 

♦ 

VV  Mean  of  AOA  Control  Ratings 
on  CV-4  Repetitions  ( 1 7. 5 Turns) 

2.49 

2.41 

.25 

* 

V  H-  Mean  of  Overall  Ratings 

on  CV-4  Repetitions  ( 17.5Turns) 

2.41 

2.36 

.10 

* 

V V  -  Miradeslip  Ratings  +■ 

2.25 

2.11 

.64 

* 

V,„-  1*'  Repetition  Overall 

Rating  on  CV  -4(21 .5Tums) 

2.1 

2.11 

.01 

** 

V  ,,  -  Mean  of  Altitude  Deviations 
on  CV-4  Repetitions  (2 1.5 Turns) 

113.95 

164.77 

2.73 

.24 

\  12-Mean  of  AOA  Control 

Ratings  on  CV-4  (2 1 . 5  Turns) 

2.63 

2.53 

.39 

* 

V, Mean  of  Overall  Ratings 
on  CV-4  Repetitions  (2 1 . 5 Turns) 

2.47 

2.35 

.43 

* 

V, , -  ( "  Repetition  Overall 

Hating  on  CV-5(  !7.5Tums) 

2.28 

2.H 

.90 

* 

V  |  Mean  of  Altitude  Deviations 
on  CV  -51 1  7.5  Turns) 

97.02 

114.35 

.35 

* 

Talile  (.'-3.  (Continued) 


Yuriiihtr 

Mnn 

•**P 

Mnn 

runt 

K  Value 
dtf  1,38) 

Power 

Ealimntf 

\  )lf  -Mean  of  AOMionlrol  Katinas 
mi  ( A -5 1 1 7  ..  Turns! 

2.96 

2.16 

13.68*** 

.9.3 

\  ,  -  -  Mean  ui  (  K  i-rall  Repetition 

Kalinas  on  (  .V  -5  ( 1  7. 5  Turns! 

2.59 

2.44 

1 .63 

.12 

\  lM-  (  A  -5< rradeslip  Ratings  + 

2.6 

2.4 

1.58 

.11 

V  i»j-  1H|  ion  Overall 

Hating  on  CV-5  (2 1 .5  Turns! 

2.3 

2.26 

1.63 

.12 

\  Mean  of  Altitude*  Deviations 

on  (  A  -5(21  .5  Turns) 

82.98 

115.2 

1.44 

.09 

V  21  -  Mean  of  AC) A  Control 

Ratings  on  (A  -.>(21 .5  Turns! 

2.9 

2.52 

5.25*** 

.52 

V  2  > ~  Mean  of  Overall  Repetition 

Ratings  on  (A  -5(21 .5  Turns! 

2.64 

2.59 

.21 

* 

\  j  ,-  1“'  Repetition  Overall 

Hating  onCY  -6(  1 7. 5  Turns) 

2.65 

2.64 

.01 

* 

\  -  Mean  of  Altitude  Deviations 

on  CY-61  1 7.5Turns) 

63.24 

94.0 

2.65 

.2.3 

\  Mean  of  AOA  Control 

Matings  on  C V-6 1 1 7. 3 Turns) 

2.91 

2.86 

/ 

.15 

* 

V  2i,- Mean  of  Overall  Repetition 

Ratings  on  CY-6(  17. 5  Turns! 

2.84 

2.75 

.28 

* 

\  2- -  CV-<>  Oradeslip  Ratings  +- 

2.5 

2.. 58 

.20 

** 

V  2h-  I"  Repetition  Overall 

Mating  on  C  V-6 (21 .3  Turns) 

2.85 

2.59 

1.79 

.14 

Ypt-  Mean  of  Altitude  Deviations 
on  CV  -(>(21 .5  Turn*) 

59.2 

78.1 

1.03 

.02 
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Tat»l<‘ (J-3.  ( Continued ) 


Variable 


Mean 


exp 


Mean  F  Value  Power 

r°m  <df  1,38)  Estimate 


\  Ml- Mean  of  \()  A  Control  Ratings 

on  CV-(»|2 1 .5  Turns)  2.97  2.8  1.47  .10 

V  -  Mi  an  of  Overall  Repetition 

Ratings  on  CV-6  (2 1 .5  Turns)  2.95  2.77  1.30  .08 


+  Denotes  variables  derived  from  student  mission  grades! ip  data;  all  other  variables  derived  from  BAM  student 
{►eHormam  e  questionnaire  data. 

*  1  )enotes  variables  for  which  l»etween  group  variances  too  small  to  compute  power  estimate. 

**  Denotes  variables  for  which  power  estimate  not  computed  due  to  better  control  group  performances. 

***  p<  .05. 


Ttihlr  Cl.  Variables,  f  > 

roup  Menus  and  Statistical  Power  Kslimuli 
for  Aireraft  30°  URK  Data 

■h(  alpha  ~ 

.05) 

Variable 

M,,u,«p 

Mean 

coni 

F  Value 
<df  =  1,38) 

Power 

Estimate 

\  (-Highest  Hepetition  Hating 
obtained  in  A/C 

.4.25 

3.05 

1.03 

.03 

\  2-  Numlierof  Repetitions  to 
Profieieney 

1.55 

2.25 

5.07*** 

.50 

\  t-  1st  Repetition  Overall 

Kali  upon  CV-4 

1.5 

1.25 

1.27 

.08 

\  t-  Mean  of  Airspeed  Deviations 
on  CV  -  V  Repetitions 

9.94 

13.1 

.89 

* 

\  Mean  of  Dive  Angle  Deviations 
on  C\  -  f  Repetitions 

2.55 

3.47 

2.36 

.21 

\  Mean  of  ( Kcrall  Repetition 

Ratings  on  CV-4  Rrp*titions 

2.19 

1.96 

2.01 

.16 

V --t!\ -4  (iraileslips  Ratings  + 

1.9 

1.8 

.29 

* 

1  !»l  Repetition  Overall 

Hut  Hint  hi  t.V-6 

2.2 

2.13 

.08 

* 

V.,-  Mean  of  Airspeed  Deviations 
on  (  A  -6  Repetitions 

3.62 

3.44 

.01 

** 

V  ln-  Mean  of  Dive  Angle  Deviations 
mi  (A  -ti  Repetitions 

1.83 

1.39 

.60 

** 

V  1 1  -  Mean  oMK erall  Repetition 
Ratings  on  CV-6  Repetitions 

2.62 

2.59 

.03 

* 

\' '  |  j  - <  A  -(>  (iradeslip  Ratings  + 

2.35 

2.35 

.00 

* 

+  Denote*  variables  derived  from  student  mission  grades) ip  data;  ail  other  variables  derived  Irom  BAM  student 
performance  questionnaire  data. 

*  Denote*  variables  for  which  laMween  group  variances  tiN>  small  to  compute  power  estimate. 

**  Denotes  variables  for  which  fsiwer  estimate  not  computeil  due  to  belter  control  group  jierformam  es. 

***p"  05 


X\ 


Tnlilr  ('.  5.  Variables,  Group  Mean*  and  Statistical  Power  Estimate*  (alpha*  -  .05) 
for  Aircraft  20°  I)RR  Data 


Variable 

Mean 

exp 

Mean 

cont 

F  Value 

<df  =  1,38) 

Power 

Estimate 

V ,  -  Highest  Repetition  Rating 
obtained  in  A/( ! 

3.4 

3.15 

1.55 

.n 

V  2  -  Number  of  Repetitions  to 

Proficiency 

1.35 

1.35 

.00 

* 

V;1-  1st  Repetition  Overall 

Rating  on  CV-5 

2.05 

1.84 

.84 

♦ 

V Mean  of  Airspeed  Deviations 
on  CV-5  Repetitions 

5.5 

8.29 

2.69 

.24 

V:>-  Mean  of  Dive  Angle  Deviations 
on  CV-5  Repetitions 

1.71 

1.49 

.45 

** 

Vft-Mean  of  Overall  Repetition 

Ratings  on  CV-5  Repetitions 

2.59 

2.29 

4.29*** 

.44 

V7-CV-50radeslips  Ratings  + 

2.3 

2.11 

1.21 

.07 

Vs-  1st  Repetition  Overall 

Rating  on  CV-6 

2.4 

2.24 

.46 

♦ 

V  .)-  Mean  of  Airspeed  Deviation 
mi  DV-f)  Repetitions 

4.93 

4.91 

.01 

** 

V  Mean  of  Dive  Angle  Deviations 
nnCV-f>  Repetitions 

1.45 

1.04 

.63 

** 

V 1 1  -  Mean  of  Overall  Repetition 

Ratings  on  CV-6  Repetitions 

2.77 

2.61 

.94 

★ 

V  CV-6  Gradeslip Ratings  + 

2.45 

2.26 

1.53 

.11 

+  Denotes  variables  derived  from  siudent  mission  gradeslip  data;  all  other  variables  derived  from  BAM  student 
performance  questionnaire  data. 

*  Denotes  variables  for  whic  h  la-tween  group  variances  loo  small  to  compute  power  estimate. 

**  Denotes  variables  for  which  |siwer  estimate  not  computed  line  to  belter  control  group  performances. 

***  p <05. 
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Table  C-6.  Variable*,  Grouo  Meant  and  Statistical  Power  Estimates  (alpha  =  .05) 

for  Aircraft  Target  Tracking  Data 


Variable 

Mean 

exp 

Mean 

ront 

F  Value 
<df  =  1,38) 

Power 

Estimate 

V  |  -  Highest  Repetition  Rating 
obtained  in  A/C 

2.7 

2.53 

.67 

* 

V  2  -  Number  of  Repetitions  to 

Proficiency 

1.1 

1.32 

1.49 

.10 

V  .i-  1st  Repetition  Overall 

Rating 

2.3 

2.0 

2.41 

.21 

V  Mean  of  Overall  Repetition 

Ratings 

2. 48 

2.18 

3.13 

.30 

Vs -Grarleslip  Ratings  + 

2.21 

1.95 

3.31 

.32 

+  Denotes  variables  derived  from  student  mission  gradeslip  data;  all  other  variables  derived  from  BAM  student 
performance  questionnaire  data. 

Denotes  variables  for  which  between  group  variances  too  small  to  compute  power  estimate. 
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APPENDIX  I):  PH  ASK  II  TASK  VAKIABI.KS.  GROUP  MEANS.  AND 
AIRCRAFT  DATA  K  VALUES  WITH  STATISTICAL  POWER  ESTIMATE 


:»<> 


Tahir  I)  I  \  ariablt'H,  Group  Mran*  ami  Slati*liral  hiv»rr  h  i  alpha  05) 

for  Aircrall  Sorlir  H  AM  - 1  I  I A  HhIh 


\  urialtli' 


\  \  -  Mran  \umi»n  « >f  I  :rnr>  !  ,r>t 

<  hrr  1.1  \  K«m:r  I 

\  j  -  Mran  Innr  I )i  v  iatmn  <  >\  <*r 
H< ii it« *  (  !hrr  kpoini  - 

\  t-Tirnr  1  >n latimon  I  a>l 
( .hrrkpoml 

\  j •  \1  (‘tin  Altitude  1  >r\ iation 
()\rr  I  I  N  Kontr  I  .res 

\  ,  -  Mran  \  irsprrd  I  )r\  iah«m 

<  )\rr  1 .1  \  Untile  1  r$;s 

\  (t-  Mran  <  ,uur*»r  ( .oniiol  Katnif.' 

<  Krr  I  1  N  RnUlr  I  .r^s 

\  rradr>hert  Katin^"  t 


rr-—‘ 

i  dill 

h  y 

Hit  1  \  1  1 

4  i 

.t'J 

.  :w 

:<»  n 

r 

Ji2 

20.  1 

it>.o 

IK 

I2K.  1 

i:.t.  i 

:<.2T 

Ui.'j 

*>.7 

10. 

2d.:> 

j.i.i 

.00 

I  :i 

2  2K 

:u 

>11  ^iade*dtp  dal.i .  .ill  . 

»di**i  \ai  i .»l •!**"» « 1 » - :  iv 

•■d  Jr«*m  1  l  \  "tiidc 

Poi *rr 
Kitimalc 


p«*rl<M  mail'  ••  ■  j*i**—l mim.M r**  data 

I  >r n* »!»■".  v  iiri.tl •!»•>  f « >i  w huh  l»!*twt*rii  ^r«uip  s  ai  ian«  lo<»  -in.ill  i< • .  »‘"i  nn,i'r 

**  !  lemite**  v  an.ilile*.  t« *i  \thieh  |ki\*«m  estimate*  1 1> >t  «  <  on  pitted  dm*  I • »  heUei  i  «»ntrol  k>r«»iip  m.n  •  «•- 

nr, 


Table  l)  2.  Variable**  Group  Mean*  untl  Stathtical  Power  K«t im at «*n  (alpha  =  .05) 
for  Aircraft  Sortie  SA- 1  IX!S  Duta 


Variable 

Mean 

exp 

Mean 

font 

V  V.lur 
uH  1.31 1 

Power 

Kfltimalr 

\  |  -  VI ran  Number ol  1  tmr*  1  a»*l 
( her  l  I  A  Route  1  rg> 

.21 

.  w> 

H 

*  * 

\  Mean  lime  1  )e\  iatinti  1  Kn 
Route  ( !heek|K»iut* 

]  I.2H 

H.H 

** 

\  t- Time  Dei  ration  on  l.a.-l 
(iheekpoiut 

H.  t 

.(>7) 

*  * 

\  ,  -  Mran  Altitude  Deviation 
t her  1  l  V  Route  1  .eg* 

27<>  1 

20H.  7 

1.71 

*  * 

V  ,-Mran  \ir>perd  I)r\ lalnm 
(her  II A  Koule  1  .eg* 

I:.*.: 

H  7 

1  7H 

*  * 

V  Mean  <  amrsr  ( imitrof  Rating 
<  Krr  1 .1  A  Route  1  eg* 

2  ><i 

J.l> 

22 

*  * 

V  :-l  iradr*beel  Haling*  ♦ 

L'.ltll 

S.  1 

*  * 

l)#*nolr*»  t  (triable.*  «l*  innl  |n»n»  ■Hiiiiiml  rni>si«iti  gia«lr*!ij»  < l.n.t :  all  "ihri  variable*  derived  trout  1  l  N  Mitdeni 
pertorman  r  ipir*tinmtair'-  ilal.t 

*  |)eiN»lr*  variable*  for  who  h  between  group  \ai  iuih  «■*  I •  m •  *mall  to  «  •  »itij Mil**  |M»wer  e*ti»nute. 

*  *  I \  unable*  lor  win*  l*  power  e*iimate*  not  i  oinputed  due  t. >  !*#•! ir*i  <  •miml  group  perfortnanr  »*>. 
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